refactor: rewrite block processor and node to use signet-storage by prestwich · Pull Request #79 · init4tech/node-components

prestwich · 2026-02-15T21:33:11Z

Summary

Rewrites the block processor to use signet-hot / signet-storage-types instead of signet-db / signet-node-types
Rewrites signet-node to use UnifiedStorage and signet-rpc-storage instead of reth's ProviderFactory and signet-rpc
Processor becomes a stateless executor: reads from hot storage via HotKv, runs EVM, returns ExecutedBlock
Node drives block extraction via Extractor + ExtractableChainShim and shares state with RPC through BlockTags and broadcast::Sender

Block Processor Changes

Dependencies (Cargo.toml):

Removed: signet-db, signet-node-types, reth-exex, reth-node-api
Added: signet-hot, signet-storage-types

Processor (processor.rs):

Generic Db: NodeTypesDbTrait → H: HotKv
ru_provider: ProviderFactory → hot: H
state_provider_database() → revm_state() using RevmRead::at_height()
on_host_commit() (monolithic loop) → process_block() (single block → ExecutedBlock)
commit_evm_results() removed — processor no longer writes to storage
New build_executed_block() handles type conversion (receipts, events, headers)

Node Changes

Dependencies (Cargo.toml):

Removed: signet-db, signet-node-types, signet-rpc, reth-db, reth-db-common, reth-chainspec
Added: signet-storage, signet-rpc-storage, signet-extract, signet-storage-types, signet-hot, trevm

Node struct (node.rs):

Db: NodeTypesDbTrait → H: HotKv
ru_provider: ProviderFactory → storage: Arc<UnifiedStorage<H>>
New tags: BlockTags and notif_tx: broadcast::Sender<NewBlockNotification> for RPC state sharing
Removed NodePrimitivesProvider and CanonStateSubscriptions trait impls
New process_committed_chain() drives extraction + processing loop
New notify_new_block() sends notifications on broadcast channel
update_block_tags() replaces update_canon_heights() / update_canon_state()

Builder (builder.rs):

NotADb / with_db() / with_factory() → NotAStorage / with_storage(Arc<UnifiedStorage<H>>)
Genesis loading via HistoryRead::has_block() + HistoryWrite::load_genesis() + UnsafeDbWrite::commit()

RPC (rpc.rs):

signet_rpc::RpcCtx → signet_rpc_storage::StorageRpcCtx with shared Arc<UnifiedStorage>, BlockTags, broadcast::Sender

RPC Storage (rpc-storage/src/config/ctx.rs):

StorageRpcCtx::new() now accepts Arc<UnifiedStorage<H>> for shared ownership with the node

Metrics (metrics.rs):

Fixed bug: record_notification_received() was calling inc_notifications_processed() instead of inc_notifications_received()
Added proper received/processed counter separation
Made generic over N: NodePrimitives

Breaking changes

SignetNodeBuilder no longer has with_db() / with_factory() — use with_storage() instead
The signet binary must be updated to use the new builder API

Test plan

🤖 Generated with Claude Code

crates/block-processor/src/v1/processor.rs

Replace the reth ProviderFactory-based storage with HotKv for rollup state reads. The processor becomes a stateless executor that reads from hot storage, runs the EVM, and returns ExecutedBlock. Extraction is moved to the node (PR3) to avoid lifetime issues with borrowed Extracts. - Replace Db: NodeTypesDbTrait generic with H: HotKv - Replace state_provider_database() with revm_state() using RevmRead - Remove on_host_commit() and commit_evm_results() - Add process_block() returning ExecutedBlock - Add build_executed_block() for type conversion - Remove signet-db, signet-node-types, reth-exex, reth-node-api deps - Add signet-hot, signet-storage-types deps - Remove Chain/PrimitivesOf/ExExNotification type aliases from lib.rs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace reth's ProviderFactory/BlockchainProvider with UnifiedStorage and swap signet-rpc for signet-rpc-storage. The node now holds Arc<UnifiedStorage<H>> and shares state with the RPC context through BlockTags (atomic block tag tracking) and broadcast::Sender (new block notifications). Key changes: - StorageRpcCtx accepts Arc<UnifiedStorage<H>> for shared ownership - Node struct uses HotKv generic instead of NodeTypesDbTrait - Block processing uses signet-extract's Extractor + ExtractableChainShim - Genesis loading via HistoryWrite::load_genesis + UnsafeDbWrite::commit - Fix metrics bug: record_notification_received now correctly increments the received counter instead of the processed counter Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace TmpDb/ProviderFactory with MemKv/UnifiedStorage in all node test infrastructure. Update signet-storage crates to v0.6.2 to fix MemKv intra-transaction read visibility. Align the cold-storage gas oracle with reth's GasPriceOracle by adding default_gas_price (1 Gwei), ignore_price (2 wei), and max_price (500 Gwei) to StorageRpcConfig. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

These crates are fully replaced by signet-storage, signet-hot, and signet-rpc-storage respectively. Remove 6,700+ lines of dead code and clean up 22 unused workspace dependencies. Replace reth-db tempdir_path with tempfile in signet-node-config. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Reclaim the signet-rpc name now that the old reth-backed crate is deleted. Rename directory, package, and all import paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…nner Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

prestwich · 2026-02-16T11:38:13Z

[Claude Code]

PR Review: `refactor/block-processor-hot-storage` (post-rebase onto `f105e1b`)

80 files, ~11.5k lines changed. The rebase onto PR #75's f105e1b resolved 4 previously-identified issues (SubCleanerTask thread leak, reward percentile overflow, dead code, doc test ignore). The remaining findings are below.

Build Breakage

crates/rpc/tests/eth_rpc.rs:625 — The rebase renamed the crate from signet-rpc-storage to signet-rpc, but the test still references signet_rpc_storage::SyncStatus. This won't compile.

Bugs

1. `log_index` is per-receipt instead of per-block

crates/rpc/src/interest/kind.rs:63

log_index: Some(log_idx as u64) uses the per-receipt enumeration index. Per Ethereum spec, logIndex must be the global index across all logs in the block. A block with two receipts of 3 logs each produces [0,1,2,0,1,2] instead of [0,1,2,3,4,5]. Fix: maintain a running counter across the outer flat_map.

2. Hash/height mismatch in `update_highest_processed_height` (pre-existing)

crates/node/src/node.rs:426-441

ExExEvent::FinishedHeight is sent with adjusted_height (N-1) but hash from the header at height N. The hash should come from the block at adjusted_height, not finalized_host_height. Worth fixing since this code is being refactored.

Should Fix

3. Panic in `serve_ipc` where `?` should be used

crates/node/src/serve.rs:217 — to_name(name).expect("invalid name") should use ? since the function returns eyre::Result.

4. Two reader transactions per `process_block` — consistency risk

crates/block-processor/src/v1/processor.rs:173-176, 96 — run_evm opens one reader for get_header, then revm_state() opens a second. A write between them means different snapshots. Consider opening one reader and threading it through.

5. Double hash computation

crates/block-processor/src/v1/processor.rs:180, 245 — Line 180: into_inner() discards the existing hash, then SealedHeader::new() wraps with an empty OnceLock. Line 245: header().clone().seal_slow() re-hashes an already-sealed header. Consider propagating the known hash.

6. Visibility over-exposure

File	Items	Issue
`rpc/src/config/ctx.rs`	`storage()`, `cold()`, `hot_reader()`, `constants()`, `chain_id()`, `config()`, `acquire_tracing_permit()`, `tx_cache()`, `revm_state_at_height()`	`pub` but only used within crate — should be `pub(crate)`
`rpc/src/eth/error.rs:34`	`EthError::into_string()`	`pub` but zero callers — remove
`rpc/src/debug/error.rs:38`	`DebugError::into_string()`	`pub` but zero callers — remove
`node/src/node.rs:269`	`on_notification()`	`pub` but only called within crate
`node/src/node.rs:446`	`on_host_revert()`	`pub` but only called within crate
`node/src/rpc.rs:23`	`start_rpc()`	`pub` but only called from `node.rs`
`node/src/serve.rs:49-57`	`ServeConfig` fields	struct is `pub(crate)` but all fields are `pub`

Style / Nits

7. Supported methods listed under "Unsupported" comment

crates/rpc/src/eth/mod.rs:65-90 — syncing, gasPrice, maxPriorityFeePerGas, feeHistory, createAccessList are fully implemented but placed after the // Unsupported methods comment.

8. Imports from same crate not grouped

6 files have separate use signet_hot::X / use signet_hot::Y that should be use signet_hot::{X, Y}:

rpc/src/config/ctx.rs:13-15 (also trevm at lines 21-22)
rpc/src/debug/mod.rs:12-13
rpc/src/debug/endpoints.rs:19-20
rpc/src/eth/mod.rs:21-22
rpc/src/signet/mod.rs:8-9
rpc/src/signet/endpoints.rs:11-12

9. Imperative loop for signet events

crates/block-processor/src/v1/processor.rs:272-285 — Three sequential for loops with a manual index counter. Could be a chained iterator with .zip(0u64..).

10. `vec![executed]` allocation per block inside a loop

crates/node/src/node.rs:308 — self.storage.append_blocks(vec![executed]) allocates per iteration. Consider batching.

11. Outdated README

crates/block-processor/README.md references deleted types but is include_str!'d into crate docs.

12. Doc issues

node/src/node.rs:24: /// Make it easier to write some args is vague on a type alias
node/src/serve.rs:80,92: serve_http and serve_ws have identical doc comments — should distinguish HTTP vs WebSocket
node/src/serve.rs:159: Result<JoinHandle<()>, eyre::Error> instead of eyre::Result<JoinHandle<()>> (inconsistent)

13. Function-scoped imports in test file

crates/rpc/tests/eth_rpc.rs:140, 414-415, 821 — should be hoisted to module level. SealedHeader at line 821 is redundant with the top-level import at line 21.

14. Missing tracing instrumentation

block-processor/src/v1/processor.rs: run_evm and build_executed_block have no #[instrument]
node/src/serve.rs:169: spawned RPC server future has no span

15. Test infrastructure

node-tests/src/context.rs:352-358: polling loop has no timeout — will hang if cold storage never reaches expected height
node-tests/src/context.rs:274-277: signet_events_in_range is defined but never called

crates/rpc/src/interest/kind.rs

crates/node/src/node.rs

crates/block-processor/src/v1/processor.rs

crates/node/src/node.rs

crates/rpc/src/eth/mod.rs

- Fix log_index per-receipt → per-block (kind.rs) - Fix hash/height mismatch in update_highest_processed_height (node.rs) - Fix serve_ipc panic: expect → ? (serve.rs) - Use seal_unchecked to avoid redundant header re-hashing (processor.rs) - Refactor imperative event loop → functional chain (processor.rs) - Add #[instrument(skip_all)] to run_evm and build_executed_block - Add TODO for two-reader consistency risk (processor.rs) - Group signet_hot imports across 6 RPC files - Move supported methods above "Unsupported" comment (eth/mod.rs) - Hoist function-scoped imports to module level (eth_rpc.rs) - Fix stale signet_rpc_storage reference (eth_rpc.rs) - Differentiate HTTP/WS doc comments, fix Result type (serve.rs) - Wrap cold storage polling loop in 30s timeout (context.rs) - Remove dead signet_events_in_range method (context.rs) - Update README to reflect new API types Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

prestwich · 2026-02-16T12:24:53Z

[Claude Code]

All review items addressed in d966f8c:

#	Finding	Status
Build	Stale `signet_rpc_storage` reference	Fixed
1	`log_index` per-receipt → per-block	Fixed
2	Hash/height mismatch	Fixed
3	`serve_ipc` panic (`expect` → `?`)	Fixed
4	Two reader transactions	TODO added (deferred)
5	Double hash computation	Fixed (line 245 `seal_unchecked`; line 180 left as-is, types differ)
6	Visibility over-exposure	Skipped (limiting scope)
7	Supported methods under "Unsupported"	Fixed
8	Import grouping (6 files)	Fixed
9	Imperative event loop	Fixed (functional chain)
10	`vec![executed]` allocation	Left as-is (no single-block API)
11	Outdated README	Fixed
12	Doc issues (serve.rs, node.rs)	Fixed
13	Function-scoped test imports	Fixed
14	Missing tracing instrumentation	Fixed (`run_evm`, `build_executed_block`)
15	Test infra (timeout, dead code)	Fixed

Verification: clippy clean on all 4 crates (all-features + no-default-features), cargo +nightly fmt, 35/35 RPC tests pass.

prestwich requested a review from a team as a code owner February 15, 2026 21:33

prestwich commented Feb 15, 2026

View reviewed changes

crates/block-processor/src/v1/processor.rs Show resolved Hide resolved

prestwich changed the title ~~refactor: rewrite block processor to use signet-hot storage~~ refactor: rewrite block processor and node to use signet-storage Feb 15, 2026

prestwich and others added 7 commits February 16, 2026 06:34

refactor: rename signet-rpc-storage to signet-rpc

d522303

Reclaim the signet-rpc name now that the old reth-backed crate is deleted. Rename directory, package, and all import paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: restore RUN EVM ascii art banner

3474485

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: resolve clippy empty_line_after_doc_comments warning in ascii ba…

809d6d7

…nner Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

prestwich force-pushed the refactor/block-processor-hot-storage branch from ff4e661 to 809d6d7 Compare February 16, 2026 11:38